Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Conversation

@namchuai
Copy link
Contributor

@namchuai namchuai commented Aug 27, 2024

Describe Your Changes

  • Add engine install with hardware detection.

Screenshots

Engine init cortex.llamacpp on windows. For this machine, AMD Ryzen 5950x, it choose AVX2, cuda version 12.0

image

Engine init cortex.onnx

image

Engine init cortex.tensorrt-llm

image

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@vansangpfiev
Copy link
Contributor

I think we should change from init to install
The command will be something like:

cortex engines tensorrt-llm install

@namchuai namchuai force-pushed the feat/add-engine-init branch from 0145f79 to 44a0247 Compare August 28, 2024 02:38
@namchuai
Copy link
Contributor Author

I think we should change from init to install The command will be something like:

cortex engines tensorrt-llm install

Hmm, this will need to change a little bit because it goes against the design of cli application. Might have to add multiple subcommand though.

@namchuai namchuai force-pushed the feat/add-engine-init branch from 44a0247 to 287c750 Compare August 28, 2024 03:39
@vansangpfiev vansangpfiev changed the title feat: add engine init cli feat: add engine install cli Aug 28, 2024
@namchuai namchuai merged commit de5716c into feat/cortex-engine Aug 28, 2024
@namchuai namchuai deleted the feat/add-engine-init branch August 28, 2024 04:02
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants